Storage systems store objects according to various corresponding storage architectures. Examples of storage architectures include an object storage, a file system, a block storage, and/or a database.
Each object within a storage system is associated with data and metadata. Data (also referred to herein as “object data”) includes, for example, information that is used by an end user and/or a client application. As an example, data for a Sales Application may include information about customers, products, and sales. As another example, data for a Human Resources Application may include information about employees, departments, and salaries. Metadata describes how the data is set up and/or stored. Additionally or alternatively, metadata describes characteristics about the data itself. Metadata may be used to manipulate and/or manage the objects in a storage system.
In cloud computing environments, data migration may happen for a variety of reasons. For example, policy-based tiering may trigger migration to match the cost and quality-of-service (QoS) expectations of cloud service subscribers. If a cloud user upgrades or downgrades the service, data may be moved from one storage tier to another storage tier to satisfy storage and capacity demands. Other policies may migrate data from a primary storage tier to an archival tier to reduce the storage overhead of older and less frequently accessed data.
Geo-replication and geo-location policies are another example of automated data migration rules. Geo-replication and geo-location policies trigger migration of data based on the location of cloud service subscribers. Data may be moved from one geographic location to another geographic location if the cloud user moves or expands to a different geographic location. Migrating data to be geographically closer to a user may decrease access times, improve network performance, and reduce storage costs.
In other examples, data may be migrated due to updated storage infrastructure. New storage offerings may reduce costs, improve performance, and render old storage technology obsolete. Cloud administrators may migrate customer data from older to newer storage technology to improve the performance of cloud service offerings provided to subscribers.
Data migration presents several challenges within the cloud environment. One challenge is that customers often expect uninterrupted access to data under a single global namespace. One approach to providing a global namespace is to maintain a mapping between object names and physical storage locations within a metadata tier of a storage system. If data within a data tier is migrated from one physical storage location to another, then the storage system updates the physical storage location of each migrated data object within the metadata tier. Thus, the same object name is mapped to the new storage location, thereby maintaining the same namespace before and after data migration. However, updating the location pointer in the metadata tier whenever data migration occurs puts a great amount of load on the metadata tier when the migration involves a large amount of data. As a result, requests requiring access to the metadata tier may suffer severe performance degradation as large data migrations are ongoing.
Another challenge of data migration is that the physical layout of data may not be consistent across different storage tiers. For example, frequently accessed data may be stored on hard disk while older data may be migrated to tapes. The different storage formats add to the complexity of servicing requests to access the data as the data is being migrated. Further, the different storage formats may increase the load on the metadata tier during migration as the location pointers stored within the metadata tier are updated to the new storage format.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.